Placing spline knots in neural networks using splines as activation functions
نویسندگان
چکیده
منابع مشابه
A representer theorem for deep neural networks
We propose to optimize the activation functions of a deep neural network by adding a corresponding functional regularization to the cost function. We justify the use of a second-order total-variation criterion. This allows us to derive a general representer theorem for deep neural networks that makes a direct connection with splines and sparsity. Specifically, we show that the optimal network c...
متن کاملThe Upper Bound on Knots in Neural Networks
Neural networks with rectified linear unit activations are essentially multivariate linear splines. As such, one of many ways to measure the “complexity” or “expressivity” of a neural network is to count the number of knots in the spline model. We study the number of knots in fully-connected feedforward neural networks with rectified linear unit activation functions. We intentionally keep the n...
متن کاملA Construction of Bi orthogonal Functions to B splines with Multiple Knots
We present a construction of a re nable compactly supported vector of functions which is bi orthogonal to the vector of B splines of a given degree with multiple knots at the integers with prescribed multiplicity The construction is based on Hermite interpolatory subdivision schemes and on the relation between B splines and divided di erences The bi orthogonal vector of functions is shown to be...
متن کاملSelection of locations of knots for linear splines in random regression test-day models.
Using spline functions (segmented polynomials) in regression models requires the knowledge of the location of the knots. Knots are the points at which independent linear segments are connected. Optimal positions of knots for linear splines of different orders were determined in this study for different scenarios, using existing estimates of covariance functions and an optimization algorithm. Th...
متن کاملOn Calculating with Mplines
In computational dealings with splines, the question of representation is of primary importance. For splines of fixed order on a fixed partition, this is a question of choice of basis, since such splines form a linear space. Only three kinds of bases for spline spaces have actually been given serious attention; those consisting of truncated power functions, of cardinal splines, and of B-splines...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
- Neurocomputing
دوره 17 شماره
صفحات -
تاریخ انتشار 1997